Visibility event navigation method and system

ABSTRACT

A method of visibility event navigation includes receiving, via processing circuitry of a client device, a first visibility event packet from a server, the first visibility event packet including information representing 3D surface elements of an environmental model that are occluded from a first viewcell and not occluded from a second viewcell, the first and second viewcells representing spatial regions of a specified navigational route within a real environment modeled by the environmental model. The method also includes acquiring, surface information representing the visible surfaces of the real environment at a sensor and determining, a position in the real environment by matching the surface information to the visibility event packet information. The method further includes transmitting, the position from the client device to the server and receiving a second visibility event packet from the server if the at least one position is within the specified navigational route.

FIELD

The present invention relates generally to navigation using 3Drepresentations of a given space to be navigated, and more particularlyto a method and system for streaming visibility event data to navigatingvehicles (on land, sea, air, or space), and using the visibility eventdata in 3D map-matching and other computer vision navigation methods.

BACKGROUND

The “background” description provided herein is for the purpose ofgenerally presenting the context of the disclosure. Work of thepresently named inventor, to the extent it is described in thisbackground section, as well as aspects of the description which may nototherwise qualify as prior art at the time of filing, are neitherexpressly or impliedly admitted as prior art against the presentdisclosure.

Precision navigation in obstacle-strewn environments such as the urbancanyon or indoors is challanging. Typically GPS does not provide theprecision required to navigate in obstacle rich environments. Also GPSreception requires line of sight access to at least four satellites,which is often not available in the urban canyon because of occlusion ofthe sky by buildings. Additionally, GPS radio signals are relativelyweak, and can be jammed or spoofed (replaced by a false data stream thatcan mislead the targeted navigational system).

Higher-power radio navigation methods such as eLORAN may be lesssusceptible to jamming near the transmitter. Otherwise these methodsinherit most of the vulnerabilities of GPS radio navigation, includingsusceptibility to denial of service by attack on the fixed transmitters.The precision of eLORAN localization is significantly less than GPS, andthe global availability of eLORAN service has been significantly limitedby the recent decision by the United Kingdom to discontinue eLORANservice in Europe.

3D map-matching is a navigational basis that is orthogonal to GPS andeLORAN navigation and consequently does not suffer from the samelimitations and vulnerabilities. Early 2.5D map matching systems such asTERCOM (Terrain Contour Matching), were effectively employed in cruisemissile navigation prior to GPS. The ability to pre-acquire detailed 3Denvironmental data has increased exponentially since the time of TERCOM.Moreover, commodity sensors are now available which generate real-timepoint-clouds that could potentially be matched to the pre-acquired 3Denvironmental data to provide rapid, precise localization in many GPSdenied environments.

However, two problems have slowed the general adoption of efficient 3Dmap-match solutions. First, because the 3D environmental data sets areso large, it can be difficult to transmit and maintain them overexisting networks using conventional data delivery approaches. Second,processing of these massive 3D datasets by 3D map-matching algorithmscan be very inefficient because the matching algorithm is typicallyforced to process a large amount of occluded data that is irrelevant tothe immediate 3D map-match localization solution. This is especiallytrue in densely occluded natural terrains, indoors, or within the urbancanyon, where buildings make most of the surfaces of the environmentoccluded from any small region.

SUMMARY

In an exemplary aspect, a method of visibility event navigation includesone or more visibility event packets located at a server, the one ormore visibility event packets including visibility event packetinformation representing 3D surface elements of a geospatial model thatare occluded from a first viewcell and not occluded from a secondviewcell, the first and second viewcells representing spatial regions ofa navigational route within a real environment modeled by the geospatialmodel. The method of visibility event navigation also includesreceiving, via processing circuitry of a client device, at least onevisibility event packet of the one or more visibility event packets fromthe server, detecting, via the circuitry, surface informationrepresenting one or more visible surfaces of the real environment at asensor in communication with the client device, and calculating, via thecircuitry, at least one position of the client device in the realenvironment by matching the surface information to the visibility eventpacket information corresponding to a first visibility event packet ofthe one or more visibility event packets. The method of visibility eventnavigation further includes, transmitting, via the circuitry, the atleast one position from the client device to the server, and receiving,via the circuitry, at least one second visibility event packet of theone or more visibility event packets when the at least one position iswithin the navigational route at the client device from the server.

In some exemplary aspects, a method of visibility event navigationincludes one or more visibility event packets located at a server,including information representing 3D surface elements of a geospatialmodel that are occluded from a first viewcell and not occluded from asecond viewcell, the first and second viewcells representing spatialregions of a navigational route within a real environment modeled by thegeospatial model. The method of visibility event navigation alsoincludes prefetching, via processing circuitry of the server, a firstvisibility event packet of the one or more visibility event packets to aclient device, receiving, via the circuitry, at least one position ofthe client device in the real environment at the server, andtransmitting, via the circuitry, a second visibility event packet of theone or more visibility event packets to the client device when the atleast one position is within the navigational route.

In certain exemplary aspects, a method of visibility event navigationprefetch includes at least one partial visibility event packet includinga subset of a complete visibility event packet, the complete visibilityevent packet including information representing 3D surface elements of ageospatial model occluded from a first viewcell and not occluded from asecond viewcell, the first and second viewcells representing spatialregions of a navigational route within a real environment modeled by thegeospatial model. The method of visibility event navigation prefetchfurther includes receiving, via processing circuitry of a server,information at the server from a client device representing anorientation of a sensor located at the client device, the sensoracquiring information representing the visible surfaces of the realenvironment, and transmitting, via the circuitry, the at least onepartial visibility event packet from the server to the client device,wherein the at least one partial visibility event packet intersects amaximal view frustum including a volume of space intersected by the viewfrustum of the sensor during movement of the client device in the secondviewcell.

In some exemplary aspects, a method of visibility event navigationincludes at least one partial visibility event packet located at aserver, the at least one partial visibility event packet including asubset of a complete visibility event packet, the complete visibilityevent packet including visibility event packet information representing3D surface elements of a geospatial model occluded from a first viewcelland not occluded from a second viewcell, the first and second viewcellsrepresenting spatial regions of a navigational route within a realenvironment modeled by the geospatial model. The method of visibilityevent navigation further includes transmitting, via processing circuitryof a client device, surface information from the client device to theserver corresponding to the orientation of a sensor located at theclient device, the surface information representing visible surfaces ofthe real environment, and receiving, via the circuitry, the at least onepartial visibility event packet at the client device from the serverincluding a subset of the visibility event packet information thatintersects a maximal view frustum, wherein the maximal view frustumincludes a volume of space intersected by a view frustum of the sensorduring movement of the client device in the second viewcell.

In some exemplary aspects, a method of visibility event navigationincludes a first visibility event packet of one or more visibility eventpackets from a server, the one or more visibility event packetsincluding visibility event packet information representing 3D surfaceelements of a geospatial model that are occluded from a first viewcelland not occluded from a second viewcell, the first and second viewcellsrepresenting spatial regions of a navigational route within a realenvironment modeled by the geospatial model. The method of visibilityevent navigation also includes, detecting, via processing circuitry of afirst client device of a plurality of client devices, surfaceinformation representing visible surfaces of the real environment at asensor in communication with the first client device of the plurality ofclient device, calculating, via the circuitry, at least one position ofthe first client device of the plurality of client devices in the realenvironment by matching the surface information to the visibility eventpacket information, and transmitting, via the circuitry, the at leastone position from the first client device of the plurality of clientdevices to the server. The method of visibility event navigation furtherincludes receiving, via the circuitry, at least one second visibilityevent packet of the one or more visibility event packets at the firstclient device of the plurality of client devices from the server whenthe at least one position is within the navigational route, detecting,via the circuitry, position information representing the position of atleast one second client device of the one or more client devices in thereal environment at the sensor, and transmitting, via the circuitry, theposition information from the first client device of the plurality ofclient devices to the server.

In certain exemplary aspects, a method of visibility event navigationprefetch includes a first visibility event packet of one or morevisibility event packets, the one or more visibility event packetsincluding visibility event packet information representing 3D surfaceelements of a geospatial model that are occluded from a first viewcelland not occluded from a second viewcell, the first and second viewcellsrepresenting spatial regions of a navigational route within a realenvironment modeled by the geospatial model. The method of visibilityevent navigation prefetch further includes receiving, via processingcircuitry of a server, at least one position of a client device in thereal environment at the server from the client device, and transmitting,via the circuitry, a second visibility event packet of the one or morevisibility event packets when the at least one position of the clientdevice is within the navigational route and a fee has been paid by anoperator of the client device.

In some exemplary aspects, a method of visibility event navigationincludes one or more visibility event packets located at a server, theone or more visibility event packets including visibility event packetinformation representing 3D surface elements of a geospatial model thatare occluded from a first viewcell and not occluded from a secondviewcell, the first and second viewcells representing spatial regions ofa navigational route within a real environment modeled by the geospatialmodel. The method of visibility event navigation also includesreceiving, via processing circuitry of a client device, at least onevisibility event packet of the one or more visibility event packets fromthe server, detecting, via processing circuitry of the client device,surface information representing one or more visible surfaces of thereal environment at a sensor in communication with the client device,calculating, via the circuitry, at least one position of the clientdevice in the real environment by matching the surface information tothe visibility event packet information corresponding to the firstvisibility event packet of the one or more visibility event packets, andtransmitting, via the circuitry, the at least one position in the realenvironment from the client device to the server The method ofvisibility event navigation further includes receiving, via thecircuitry, at least one second visibility event packet from the serverof the one or more visibility event packets at the client device fromthe server, calculating, via the circuitry, at least one deviation ofthe ground-truth 3D structure from the corresponding environment modeledby the geospatial model using the surface information and the visibilityevent packet information and transmitting, via the circuitry, the atleast one deviation from the client device to the server.

In certain exemplary aspects, a visibility event navigation systemincludes a server and at least one client device located in a realenvironment and in communication with the server. The at least oneclient device includes processing circuitry configured to detect surfaceinformation representing one or more visible surfaces of the realenvironment at one or more sensors in communication with the at leastone client device, and calculate at least one position of the at leastone client device in the real environment by matching the surfaceinformation to visibility event packet information including a firstvisibility event packet of one or more visibility event packetsrepresenting 3D surface elements of a geospatial model that are occludedfrom a first viewcell and not occluded from a second viewcell, the firstand second viewcells representing spatial regions of a navigationalroute within the real environment and modeled by the geospatial model.The processing circuitry is further configured to transmit the at leastone position of the client device to the server and receive a secondvisibility event packet of the one or more visibility event packets fromthe server when the at least one position is within the navigationalroute.

The foregoing paragraphs have been provided by way of generalintroduction, and are not intended to limit the scope of the claims. Thedescribed aspects, together with further advantages, will be bestunderstood by reference to the following detailed description taken inconjunction with the accompanying drawings.

BRIEF DESCRIPTION OF THE DRAWINGS

A more complete appreciation of the disclosure and many of the attendantadvantages thereof will be readily obtained as the same becomes betterunderstood by reference to the following detailed description whenconsidered in connection with the accompanying drawings, wherein:

FIG. 1 is an exemplary illustration of a visibility event navigationsystem, according to certain aspects.

FIG. 2A is an exemplary illustration of an exemplary viewpoint and acorresponding exemplary view frustum having a 90 degree horizontal fieldof view, according to certain aspects;

FIG. 2B is an exemplary illustration of a conservative current maximalviewpoint extent (CCMVE) of penetration into a viewcell from a knownposition after 166 ms of elapsed time using the exemplary view frustumhaving a 90 degree horizontal field of view, according to certainaspects;

FIG. 3 is an exemplary illustration of a conservative from-subregionfrustum, according to certain aspects;

FIG. 4 is an exemplary illustration of a conservative from-subregionfrustum that results from viewpoint penetration into the viewcell over166 milliseconds for a CCMVE subregion, according to certain aspects;

FIG. 5 is an exemplary illustration of an additional angular region ofan extended frustum, according to certain aspects;

FIG. 6 is an exemplary illustration of a top-down view of a view frustumhaving a horizontal field of view of 90 degrees and undergoing rotationin the horizontal plane at a rate of 90 degrees per second, according tocertain aspects;

FIG. 7 is an exemplary illustration of a server representation of clientdevice viewpoint position and orientation, according to certain aspects;

FIG. 8 is an exemplary illustration of an architectural block diagram ofa visibility event navigation system, according to certain aspects;

FIG. 9 is an algorithmic flowchart of a visibility event navigationprocess, according to certain exemplary aspects;

FIG. 10 is an algorithmic flowchart of a visibility event navigationdifference determination process, according to certain exemplaryaspects;

FIG. 11 is an algorithmic flowchart of a visibility event navigationincident determination process, according to certain exemplary aspects;

FIG. 12 is an algorithmic flowchart of a visibility event navigationunauthorized object determination process, according to certainexemplary aspects;

FIG. 13 is a block diagram of a visibility event navigation systemworkflow, according to certain exemplary aspects;

FIG. 14 is a hardware block diagram of a client device, according tocertain exemplary aspects;

FIG. 15 is a hardware block diagram of a data processing system,according to certain exemplary aspects; and

FIG. 16 is a hardware block diagram of a CPU, according to certainexemplary aspects.

DETAILED DESCRIPTION

In the drawings, like reference numerals designate identical orcorresponding parts throughout the several views. Further, as usedherein, the words “a,” “an” and the like generally carry a meaning of“one or more,” unless stated otherwise.

Furthermore, the terms “approximately,” “approximate,” “about,” andsimilar terms generally refer to ranges that include the identifiedvalue within a margin of 20%, 10%, or preferably 5%, and any valuestherebetween.

In certain exemplary aspects, there is described a method to deliver andreceive navigational data as visibility event packets, wherein thevisibility event packets include information representing 3D surfaceelements of a geospatial model that are occluded from a first viewcelland not occluded from a second viewcell, the first and second viewcellsrepresenting spatial regions of a specified navigational route within areal environment modeled by a geospatial model. The specifiednavigational route can also be referred to as a pursuit route, pursuitroute, navigational route, route of navigation, route of navigation,desired route, desired route, specified route, specified route, and thelike, as discussed herein. As such, a navigating client device receivesat least one visibility event packet from a server.

In some aspects, the navigating client device employs sensors to acquireinformation representing the surrounding visible surfaces of the realenvironment. In certain aspects, the data is collected as a 3D pointcloud including X axis, Y axis, and Z axis values utilizing LiDAR,SONAR, optical photogrammetric reconstruction, or other sensor meansthat are known. The 3D surface information of the real environment canbe acquired by the sensor and matched to the 3D surface information ofat least one visibility event packet delivered from the server to theclient device in order to determine the position of the client device inthe real environment. In other aspects, the 3D map-matching positiondetermination is used as the sole method of determining position oraugmented with other navigational methods such as GPS, eLORAN, inertialnavigation, and the like. The 3D map-matching determination can beconfigured to employ matching of directly/indirectly acquired pointcloud data to the 3D modeled data. In some aspects, the 3D map-matchingcan also compare geometric or visual “features” acquired from theenvironment to the transmitted 3D data. In certain aspects, thetransmitted visibility event packets include 3D surface modelinformation representing the real environment as polygons, proceduralsurfaces, or other representations. Instances in which the visibilityevent packets employ procedural representation of the surfaces of themodeled environment can include the methods of employing proceduralsurface representations in visibility event packets, to reduce thetransmission bandwidth required to send the navigational data.

In some exemplary aspects, the visibility event navigation systememploys 3D map-matching of ground-truth sensor data to visibility eventpacket model data to determine a position and an orientation of a clientdevice in a real environment. In exemplary aspects, the visibility eventpackets provide a 3D representation that provides accelerated 3Dmap-matching in comparison to systems which deliver 3D model data usinga conventional proximity-based streaming. As such, proximity basedstreaming delivers many model surfaces that are occluded and thereforeirrelevant to the 3D map-match process, which generally must only matchunoccluded ground truth sensor data to the model data. Additionally,proximity based streaming methods can have higher bandwidthrequirements, because they deliver a large amount of occluded data thatis not relevant for the 3D map-match localization. This additionaloccluded data delivered by proximity-based streaming methods also tendsto make the client-side 3D map-match process less efficient, since itintroduces irrelevant data into the 3D map-match execution. In additionto delivering a potentially large amount of occluded model surfaceinformation, proximity-based streaming methods can fail to deliverpotentially useful, unoccluded model surface data because it is notwithin the current proximity sphere that the server is using forprefetch. This loss of potentially useful data is exacerbated in denselyoccluded environments such as cities because the transmission of a largeamount of occluded surface model data often causes the system to reducethe proximity sphere in an effort to maintain the stream.

In some exemplary aspects, the visibility event navigation systememploys computer vision systems other than conventional 3D map-matchingto match ground-truth surfaces or features to visibility event packetmodel data representing basic surfaces or salient 3D features of themodeled environment.

In some exemplary aspects, the efficient delivery of high-precision,relevant 3D surface model data enables a localization solution that isfaster, more efficient, and more precise than simultaneous localizationand mapping (SLAM). Pure SLAM solutions do not incorporateprior-knowledge of the environment into their localization solutions butinstead depend upon effectively developing the real-time 3D model denovo, in real time. Consequently, pure SLAM is more computationallyexpensive and less precise than 3D map-matching to prior, processed(e.g., previously geo-registered), and relevant (unoccluded) 3D datadelivered by a visibility event navigational data stream. The increasedcomputational efficiency of 3D map-matching compared to SLAM-focusedlocalization can free on board sensor and computational resources tofocus on identifying, tracking, and avoiding moving objects in theenvironment, such as other client devices.

In certain exemplary aspects, visibility event navigation data iscompared to an actual ground-truth 3D representation of visible surfacesof an environment in which navigation occurs. The comparison can employ3D map-matching methods, other surface matching methods, computer visionmatching methods, and the like. In some aspects, if the comparison ofthe ground-truth sensor data and the visibility event navigation datadetermines that the ground-truth surfaces do not match the deliveredvisibility event navigation data, then data representing the differencebetween the streamed visibility event navigation data and the groundtruth sensor data is streamed to the visibility event navigation server.In some aspects, the server compares this information suggesting anapparent change in the structure of the 3D environment in a particularregion of the model corresponding to the viewcells from which thechanged surfaces would be visible. The method enables a maintainable,sustainable, scalable, and deliverable representation of earth's naturaland urban environments for the purposes of high-precision navigation.

In some aspects, the primary navigation routes represented by primaryvisibility event navigation packets delivered from the server arespecifically calculated to avoid low-altitude helicopter traffic byincorporating ADS-B (Automated Dependent Surveillance-Broadcast)information into the route planning. The server uses this dynamicinformation about helicopter and other low altitude aircraft, as well asinformation about helicopter routes and landing pads, to compute andstream routes for autonomous vehicles that avoid other low altitudeaircraft. In certain aspects, the current position of any aircraft beingcontrolled by the as determined using the visibility event navigationsystem, as well as the planned route could potentially be incorporatedinto ADS-B system or other commercial systems (e.g. commercial systemssuch as FOREFLIGHT® or GARMINPILOT®), which would make the position ofthe controlled aircraft available to other aircraft though thesenetworks.

In certain aspects, small unmanned aerial vehicles, such as small dronesoperating in a commercial application of package delivery, are givenvisibility event navigational packets corresponding to flight routesthat are intended to survey regions for wires, cables, towers, cranesand other hazards to low altitude aircraft such as helicopters. Therelevant visibility event navigation data is made available to pilotedclient devices not being controlled by the visibility event navigationsystem, for the purpose of obstacle avoidance. By transmitting theposition of the piloted client device (as determined by GPS or othernavigational means) to the server, the server can then compute real-timecollision course and issue proximity or impending collision warnings tothe client device. In some aspects, the server streams the relevantvisibility event packets so that the computation can be made on boardthe piloted client device. By prefetching all the visibility eventpackets required for an intended navigational route, a continuousconnection between the client device and the server is unnecessary toprovide the collision warning service.

In certain aspects, a server is configured to deliver visibility eventpackets to certified client devices which present authenticatedcredentials using a secure process of authentication, such as acryptographic ticket. The transmission of visibility event packets tocertified clients ensures that the client device has met certainrequirements for navigating along the intended route. In some aspects,this can allow autonomous navigation in airspace to client devicesincluding operators and airframes that meet specified requirements forsafe operation, such as airworthiness.

In some aspects, the visibility event navigational data stream is madeavailable through the provenance of an issuing navigational authority.Additionally, the visibility event navigational data stream isencrypted, authenticated, registered, and transactional. A client devicesuch as an aircraft or vehicle employing the visibility event navigationclient system can use onboard sensors to detect uncertified/unregisteredobjects operating in the vicinity of the certified/registered clientdevice, and to report the location of the intruding object to theserver. The implementation of the visibility event navigation systemcould enhance the ability of the Federal Aviation Administration (FAA),the Department of Homeland Security (DHS), or similar issuing authorityto control and monitor low-altitude airspace. As such, a sustainablerevenue stream can be generated by the issuing authority operating thevisibility event navigation stream through modest, per-trip transactionfees for autonomous or semi-autonomous client devices using thevisibility event navigation stream in the controlled airspace.

In certain aspects, the navigating client device receives visibilityevent packets for the purpose of high-precision navigation within aprescribed flight route. The client device can be required to transmitits location to the server delivering the packets at predeterminedintervals. In this instance, the server can be configured toperiodically or continuously deliver alternate visibility event packetsincluding visible surfaces that are encoded as visibility event packetsand located along one or more flight routes from a current location ofthe client device to a safe landing zone. In doing so, flight safety canbe reinforced by providing data that enhances the ability of theaircraft to make a safe landing at any point in time. In some aspects,the client device is a piloted aircraft, wherein the visibility eventdata representing an alternate route to one or more safe landing zonesis immediately available to the pilot in case of engine failure or othercircumstances that necessitate an immediate safe landing. The safelanding zone data is used by the pilot to identify reachable safelanding zones. In other aspects, the client device may be controlled byan autopilot or other system which is instructed to follow one or moreof the alternate routes supplied as a visibility event navigationalstream to a safe landing zone. The client device may be autonomous orsemiautonomous, or have an autonomous navigation system in which thevisibility event packets supply information used by 3D map-matching orother computer vision navigation systems that employ a high precision 3Drepresentation of the environment in which the navigation will occur.

In some aspects, a visibility event navigation system includes a commandwithin the client device to follow an alternative navigational route tosafe landing in case the client device loses communications with theserver. In other aspects, the server can be configured to determine ifthe location of the client device deviates from the specified flightroute, and if the client device is determined to deviate, the servertransmits a command to the client device which causes the aircraft todivert to the alternate route described by the alternate routevisibility event packets, thus making a safe landing in a predeterminedsafe landing zone. The alternate route defined by the visibility eventnavigation packets that the client device navigates along, either from adefault on-board instruction or from an instruction explicitlytransmitted from the visibility event navigational server, may notnecessarily be the closest landing zone. Instead, the alternate routemay be selected based on other safety criteria, such as proximity topeople, ground vehicles, or proximity to security-sensitive buildings ortransportation routes.

In certain aspects, the server delivers visibility event navigationpackets to visibility event navigation systems including client devicessuch as commercial aircraft, package delivery drones, and the like. Thevisibility event navigation system can deliver visibility eventnavigation packets from the server to client devices used by lawenforcement agencies, for example surveillance drones. In otherexemplary aspects, the visibility event navigation system facilitates adual-use in which commercial aircraft may optionally assist lawenforcement agencies by optionally receiving visibility event navigationpackets which define a route to an incident of threatened public safety,and optionally receive an instruction to navigate along this route tothe incident area.

In certain aspects, the transmission of navigational data as visibilityevent packets for navigation along a predetermined route includes amethod of enhanced predictive prefetching as described in copendingparent application Ser. No. 15/013,784. In some aspects, the bandwidthrequired to stream visibility event navigational packets is reduced whenthe desired navigational route, on the ground or in the air, is knownbefore packet transmission. As such, prior knowledge of the navigationalroute improves the efficiency of navigation-driven prefetch ofvisibility event packets, whether the packets are being delivered forvisualization, for example in entertainment or serious visualizationapplications, or for navigation such as visibility even navigationaldata for use in 3D map-matching or other robotic vision based navigationsystems. In some aspects, the prior knowledge of navigational intentalong a specific navigational route limits the number of visibilityevent packets that must be prefetched, since navigational uncertainty isreduced, and the ability to deterministically, rather thanprobabilistically, prefetch packets corresponding to viewcelltransitions far ahead of the navigating client is increased. In thisinstance, a deterministic prefetch approach enables efficient streamingof visibility event navigational packets over high latency and lowbandwidth networks.

FIG. 1 is an exemplary illustration of a visibility event navigationsystem 100, according to certain aspects. The visibility navigationsystem 100 describes a system for transmitting and receivingnavigational data as visibility event packets between a client device108 and a server 106. The visibility event navigation system 100 caninclude a network 102, a sensor 104, a server 106, and a client device108. The sensor 104 can be one or more sensors 104 and is connected tothe server 106 and the client device 108 via the network 102. The sensor104 can include a LiDAR sensor, a SONAR sensor, an optical sensor, andthe like. In certain aspects, the sensor 104 can be directly connectedto the client device 108, as in the case of the sensor 104 beingdirectly employed by the client device 108 for 3D map-matching, computervision surface matching methods, surface feature matching methods, andthe like. In some aspects, the sensor 104 can be utilized by circuitryof the client device 108 to identify and track moving objects in anenvironment. Further, the sensor 104 can be employed by the clientdevice 108 to acquire information representing visible surfaces of theenvironment surrounding the client device 108. In certain aspects, the3D surface information acquired by the sensor 104, via the circuitry ofthe client device 108, is matched to 3D surface information of avisibility event packet that is delivered from the server 106 to theclient device 108 in order to determine the position or location of theclient device 108 within the environment.

The server 106 can include one or more servers 106 and is connected tothe sensor 104 and the client device 108 via the network 102. The server106 includes processing circuitry that can be configured to receiveposition information of the client device 108 from the processingcircuitry of the client device 108. In some aspects, the circuitry ofthe server 106 can be configured to transmit visibility event packets tothe client device 108 when the position of the client device 108 islocated within a predetermined navigational route. In other aspects, theserver 106 can include network nodes that transmit information to one ormore client devices 108 at different positions along differentnavigational routes.

The client device 108 can include one or more client devices 108 and isconnected to the sensor 104 and the server 106 via the network 102. Theclient device 108 can include an autonomous aircraft, a semiautonomousaircraft, a piloted aircraft, an autonomous ground vehicle, asemiautonomous ground vehicle, and the like. The client device 108includes processing circuitry that can be configured to detect surfaceinformation representing visible surfaces of a real environment theclient device 108 is located in. In certain aspects, the circuitry ofthe client device 108 utilizes the sensor 104 to determine the surfaceinformation. The circuitry of the client device 108 can also beconfigured to calculate a position of the client device 108 in theenvironment by matching the surface information to visibility eventpacket information corresponding to visibility event packetsrepresenting 3D surface elements of a geospatial model that are occludedfrom a first viewcell and not occluded from a second viewcell, in whichthe first and second viewcells represent spatial regions of anavigational route with the environment that are modeled by thegeospatial mode. In some aspects, the circuitry of the client device 108can be configured to transmit the position information of the clientdevice 108 to the server 106 and receive visibility event packets fromthe server 106 when the position of the client device 108 is within thenavigational route. In certain aspects of the present disclosure, theprocessing circuitry of the server 106 can be configured to perform themethods implemented by the processing circuitry of the client device108, as described herein.

The network 102 can include one or more networks and is connected to thesensor 104, the server 106, and the client device 108. The network 102can encompass wireless networks such as Wi-Fi, BLUETOOTH, cellularnetworks including EDGE, 3G and 4G wireless cellular systems, or anyother wireless form of communication that is known, and may alsoencompass wired networks, such as Ethernet.

FIG. 2A is an exemplary illustration of an exemplary viewpoint and acorresponding exemplary view frustum having a 90 degree horizontal fieldof view 200, according to certain aspects. In certain aspects of thepresent disclosure, 3D map-matching, computer-vision based surfacemethods, feature matching methods, and the like employ 2D and/or 3Dscans of a real environment (for example, photographic or LiDAR) via asingle scan point. The scan point can also be referred to as viewpoint,as discussed herein. In some aspects, the scanning method used to obtaina representation of the real environment may have a limited directionalextent. The directional extent of the scan method can also be referredto as the scan frustum or view frustum, as discussed herein. The averageor principal direction of the scan can also be referred to as the viewdirection vector, as discussed herein. The viewpoint 202 includes acorresponding view frustum 204 that includes a view direction vector forprocessing visibility event packets. The use of view direction vectorsto compute smaller visibility event packets can be useful for streaminga fixed or limited view direction vector in real environments. In someaspects, the visibility event navigation can include acquiring 3Dsurface information of a real environment, acquired by a sensor 104 incommunication with a client device 108, and matching the 3D surfaceinformation of at least one visibility event packet to determine theposition of the client device 108 in the environment. In the generalcase, the view direction vector can be pointed in any direction for anyviewpoint within any viewcell, corresponding to a view direction vectorof a predetermined field of view. For example, FIG. 2A shows a viewpoint202, and a corresponding view frustum having a 90 degree horizontalfield of view 204.

FIG. 2B is an exemplary illustration of a conservative current maximalviewpoint extent (CCMVE) 206 of penetration into a viewcell from a knownposition after 166 ms of elapsed time using the exemplary view frustum210 having a 90 degree horizontal field of view 200, according tocertain aspects. FIG. 2B shows a top-down view of a 90 degree horizontalfield of view frustum 210 enveloping the CCMVE-5 206. FIG. 2B also showsa conservative current maximal viewpoint extent 206, CCMVE-5, ofpenetration into the viewcell 208 from a known position after 166 ms ofelapsed time. The CCMVE-5 206 is determined from a last known positionand the maximal linear and angular velocity and acceleration of theviewpoint 202. For example, for a typical 90 degree field of view suchas the 90 degree from-point frustum 204 shown in FIG. 2A, rotation ratesof the frustum 204 approach 130 to 140 degrees per second. However, a 90degree yaw ability to scan the environment is more suitable andenjoyable for the viewing of a spectator.

FIG. 3 is an exemplary illustration of a conservative from-subregionfrustum 300, according to certain aspects. FIG. 3 shows that theresulting conservative from-subregion frustum 304 is larger than thecorresponding from-point frustum 306 at viewpoint 302, even if itassumed that no view direction vector rotation has occurred, for aCCMVE-5 308 representative of predicted viewcell penetration at 166 msinto a viewcell region 310.

FIG. 4 is an exemplary illustration of conservative from-subregionfrustum that results from viewpoint penetration into the viewcell over166 milliseconds for a CCMVE subregion 400, according to certainaspects. FIG. 4 shows a resulting conservative from sub-region frustum402 that results from a CCMVE-5 404 representative of viewpointpenetration into the viewcell 406 sub-region over 166 milliseconds,together with rotation of the view direction vector 15 degrees to theright 408 or 15 degrees to the left 410 from an initial view directionvector orientation 412. In this exemplary case, assuming a maximum viewdirection rotation rate of 90 degrees per second, if the ping latencybetween the client device 108 and the server 106 is 166 ms, theresulting 30 degree rotation would represent the uncertainty of theclient device's 108 view direction vector 412, as experienced by theserver 106. Accordingly, consistent with aspects of the presentdisclosure, the server 106 can employ the extended 120 degree frustum402 (i.e., 120 degree predicted maximum from-subregion frustum) todetermine the subset of the visibility event packet data to actuallytransmit to the client device 108. This determination is made bydetermining the set of unsent surfaces of the corresponding visibilityevent packet that intersect the extended frustum.

In certain aspects, the visibility event packet data is precomputedusing the method of first-order from region visibility. The set ofsurfaces belonging to the corresponding PVS, incrementally maintainedusing the delta-PVS VE packets, that have not already been sent ismaintained using the technique of maintaining the shadow PVS on theserver 106. In some aspects, the visibility event packets areprecomputed assuming a full omnidirectional view frustum spanning 12.56steradians of solid angle. As described, in certain exemplary aspects,the server 106 can employ the extended view frustum to cull portions ofthe precomputed visibility event packet that fall outside of the maximumpossible predicted extent of the client device 108 view frustum, asdetermined from the ping latency and the maximal angular velocity andacceleration of the view frustum, as well as the maximum predictedextent of penetration of the viewpoint into the viewcell.

This method ensures that all of the potentially visible surfaces aretransmitted, while minimizing bandwidth requirements, by deferring thetransmission of visibility event packet surfaces that are not within thecurrent conservative extended frustum, or which happen to be backfacingwith respect to the conservative current maximal viewpoint extent ofpenetration into the viewcell. In exemplary aspects, there is alsodescribed a method of using reduced level-of-detail models in peripheryof extended view frustum to reduce bandwidth requirements for bufferingagainst view direction vector rotation, such as level-of-detail vs.predicted exposure durations.

The above-disclosed methods comprise determining a conservativerepresentation of the client device's 108 view frustum from the temporalreference frame of the server 106, and using this extended frustum tocull those surfaces of the corresponding visibility event packet thatcould not possibly be in the client device's 108 view frustum.Consistent with aspects of the present disclosure, all of thetransmitted surface information is represented at the highestlevel-of-detail. The visibility event packets can be encoded usinggeometric and surface models at a plurality of levels-of-detail,including a plurality of levels of geometric, texture, and other surfacedetail. In some aspects, however, the visibility event packets can betransmitted at a lower level-of-detail during periods of low bandwidthavailability, and/or high bandwidth requirement, in order to maximizethe probability that the information encoding newly exposed surfacesarrives on time, such as before the surface is actually exposed in theclient device 108 viewport. Under some conditions, a visibility eventpacket containing relatively low level-of-detail surface information caninitially be transmitted and later replaced by a visibility event packetcontaining higher level-of-detail information. This exploits the factthat certain 3D map-match systems and/or computer vision systems canhave lower precision when matching to newly exposed surfaces and moreprecision when matching to surfaces that have been exposed for a longerperiod of time. In this instance, the system has more time to convergeon a high-precision match solution for surface elements that are presentlonger. In certain aspects, sending elements of a visibility eventpacket that correspond to newly exposed surfaces for the visibilityevent navigation system 100 at low level-of-detail saves transmissionbandwidth while preserving the efficiency of client-side processing,since the level-of-detail of the transmitted information is matched tothe spatiotemporal performance profile of the computer vision or 3Dmap-matching system.

FIG. 5 is an exemplary illustration of an additional angular region ofan extended frustum 500, according to certain aspects. In certainaspects, the limited spatiotemporal performance of robotic visionsystems, including 3D map-matching navigation systems and the similarlylimited spatiotemporal performance of the human visual system, can beexploited by sending low level-of-detail surface information if surfacesfall outside the region of the extended view frustum, as determined,using one or more of the ping latency, the maximum viewpoint translationvelocity and acceleration, the maximum angular velocity, andacceleration of the view direction vector. For example, FIG. 5 shows anadditional angular region of extended view frustum 502 that spans anadditional 15 degrees 504 on each side of the extended 120 degreefrustum 402 shown in FIG. 4.

In certain aspects, the server 106 transmits surfaces that fall in thesubfrustum between 120 degrees and the maximally extended frustum of 150degrees 504 at a lower level-of-detail than the other visibility eventsurface data that fall within the 120 degree extended frustum. Thedisclosed method thus provides a region of uncertainty between 90degrees and 120 degrees of the subfrustum 506, as well as an additionalbuffer region against view direction vector rotation between 120 degreesand 150 degrees of the subfrustum 504, which may be useful if thedirectional visibility gradient, when the rate of exposure of surfacesper degree of view direction vector rotation is high, or if theavailable bandwidth has a high degree of variability, such as networkjitter. In this instance, the low level-of-detail surface informationcan potentially be replaced by a higher level-of-detail representation.

FIG. 6 is an exemplary illustration of a top-down view of a view frustumhaving a horizontal field of view of 90 degrees and undergoing rotationin the horizontal plane at a rate of 90 degrees per second 600,according to certain aspects. FIG. 6 shows a top-down view of a viewfrustum having a horizontal field of view of 90 degrees 602, andundergoing rotation in the horizontal plane at a rate of 90 degrees persecond 604 in a direction from a first region 606 toward a fourth region612. In this exemplary case, surfaces to the right-hand side of the viewfrustum 602 will undergo incursion into the rotating frustum at a firstregion 606, whereas surfaces near the left-hand extreme of the viewfrustum 602 at the first region 606 will exit the frustum 602 duringfrustum rotation. In the exemplary case shown in FIG. 6, those surfacesin the first region 606 have been in the frustum 602 for between 750 msand 1000 ms as a consequence of exposure via the second region 608, thethird region 610 and the fourth region 612 during the rotation. In thesecond region 608, for example, the surfaces have been in the frustum602 for between 500 ms and 750 ms; in the third region 610, the surfaceshave been in the frustum 602 for between 250 ms and 500 ms; and in thefourth region 612, the surfaces have been in the frustum 602 for between0 ms and 250 ms. Surfaces that have been in the frustum 602 for only abrief period of time have also been exposed to the graphical display incommunication with the client device 108 for a concomitantly briefperiod of time

FIG. 7 is an exemplary illustration of a server representation of clientviewpoint position and orientation 700, according to certain aspects.The visibility event navigation system 100 can be used for streamingvisibility event data as well as other data to a navigating clientdevice 108 such as a vehicle or aircraft. In certain aspects, the clientdevice 108 can be autonomous, semi-autonomous, on-board pilot or driveroperated, remotely operated, and the like.

The visibility event navigation system includes a server 106 that can beconfigured to employ a navigation-based prefetch scheme in which theentire specified navigational route to an intended destination ispredetermined. In certain aspects, all of the corresponding visibilityevent navigational packets, for the entirety of the navigational route,can be transmitted by the circuitry of the server 106 to the clientdevice 108 as a download prior to initiating navigation. Alternatively,all of the visibility event navigational packets can be streamed to theclient device 108 during an initial portion of the trip, in which thevisibility event packets being streamed correspond to viewcellboundaries that may be penetrated during movement of the client device108 within the specified 3D navigational route. In this instance, thevisibility event navigational server 106 and client device 108 do notrequire a network connection that is constantly available. However, ifthe specified navigational route is changed by the server 106 due totraffic, environmental changes, or any other reasons for re-routing, theprevious set of visibility event navigational packets, corresponding tothe original specified navigational route beyond the point of rerouting,may be delivered unnecessarily, making inefficient use of the availablenetwork bandwidth.

In other aspects, the visibility event navigational packets can bestreamed in a demand-paged mode in which the server 106 monitors theactual location of the navigating visibility event client device 108from self-position reports of the client device 108, sensor 104 data,and/or position reports of other participating navigating clientdevices, and streams the corresponding visibility event navigationpacket to the navigating client device 108 just before the packet isneeded.

In some aspects, a prefetch method which balances the constraints ofreducing bandwidth requirements while reliably preventing visibilityevent cache underflow on the client device 108 can be used. In thisinstance, the server 106 prefetches the visibility event packets using anavigation-predictor agent that is virtually navigating (within theserver-side environmental model) at a specified distance or a specifiedtime ahead of the client device 108 (navigating in the correspondingreal environment). As such, the server-side navigation-predictor agentcan be controlled by the server 106 to maintain a position that is aheadof the actual navigating client device's 108 corresponding position by adefined period which exceeds the round-trip time network delay betweenthe server 106 and the client device 108. In certain aspects, thedefined period is short enough to allow the intended navigational routeto be changed at any time before too much bandwidth has been committedto delivering information for a diverted route. In certain aspects, theserver 106 and/or the client device 108 can modulate velocity along theintended navigational route in order to avoid visibility event cacheunderflow. This variable-delay navigational prefetch method is describedin the copending parent application Ser. No. 15/013,784. In certainaspects of the present disclosure, the navigational prefetch method forstreaming visibility event navigational packets support 3D map-match andother computer vision based navigation in real environments.

The server representation of client position and orientation 700describes a method of navigation-based prefetch of visibility eventnavigational packets in which the area and/or volume of navigationalprediction collapses to a precise navigational route. The serverrepresentation of client position and orientation 700 includes a currentlocation of the navigating client 702, a current frustum 704. Thefrustum representing the scan frustum for a LiDAR scanner, an activesensor, a passive sensor, and the like. The position at a secondlocation 706, a second frustum 708, a pursuit route of navigation 710,deterministic prefetch 712, a future position 714, a future frustum 716,and probabilistic prefetch 718. The current position 702 follows anavigation route indicated by a future position 714. In certain aspectsof the present disclosure, the current position 702 follows two secondsbehind a future position 714. The current position 702 can provide afirst frustum 704 which indicates the field of view of the clientnavigational sensor 104 at the current position 702.

The circuitry of the server 106 modifies the position of the futureposition 714 directly. The future positions act as a virtualnavigational prefetch agent, moving ahead on the pursuit route 710 inthe modeled environment, which corresponds to the client device's 108navigational route in the real environment.

The current position 702 can represent the position of the navigatingclient device 108 in the real environment, as communicated between theclient device 108 and the server 106. In certain aspects, the positionof the client device 108 can change location as a result of the movementof the future position 714. For example, the current position 702 canmove along a specified navigational route 710 towards a second position706. The position at the second location 706 can include a secondfrustum 708 in which the field of view of the sensor 104 at a positionat the second location 706 is represented, based on the intended headingof the navigating client device 108 at position 706. In some aspects,the fields of view are processed by the circuitry of the client device108 to match the sensor data, obtained within the frustum to theportions of the environmental model, delivered by the visibility eventpackets from the server 106, that are within a corresponding virtualfrustum.

The current position 702, the position at the second location 706 andthe future position 714 travel along the intended navigational route710, which corresponds to the commands received by the circuitry of theserver 106. Additionally, the circuitry can be configured to pre-signalthe intended navigational route with respect to the location of thefuture position 714. The server representation of client position andorientation 700 can predict navigational intent with minute uncertainty,such as 166 milliseconds. The low value of uncertainty allows networklatency to be concealed when the server updates the future position 714.The future position 714 is illustrated as being the position of theserver's virtual navigational prefetch agent at a time 166 ms after acoordinated current time.

The current instantaneous state of the real navigational environment isknown to the virtual navigational prefetch agent of the server 106 witha dilution of precision that reflects the round trip time (RTT) betweenthe server 106 and the sensor 104 utilized to report locations ofobjects in the real environment.

The position and orientation of the future position 702 is known to theserver 106 with dilution of precision that reflects the 166 ms RTTbetween the client device 108 and the server 106. The circuitry of theserver 106 can determine the position and orientation of the futureposition to within ½*RTT, however, any visibility event packettransmitted from the server 106 to the client device 108 will be delayedan additional ½*RTT, making the effective precision bounds limited toRTT. Consistent with certain aspects of the present disclosure, it isassumed that the dilution of precision is determined by the full RTT.

In some aspects, the circuitry of the server 106 determinesrepresentations of the future position 714 and transmits therepresentations to the client device 108. The representations can beless than RTT old (time elapsed from the current time) and have adilution of precision that is a function of (RTT—elapsed time). Theserver 106 representation of the future position 714 includes a futurefrustum 716 and corresponding probabilistic prefetch 718 which can beutilized to determine a predictive route of uncertainty. This predictiveroute of uncertainty utilizes the probabilistic prefetch 718 in whichthe future position 714 and orientation states less than RTT time haselapsed from the current time is shown as a dashed trajectory with afuture frustum 716, otherwise referred to as a superimposed navigationalcone of uncertainty, reflecting the server's 106 effective uncertaintyof position and/or orientation of the future position 714.

As such, when the RTT exceeds the elapsed time, then the server's 106representation of the current position 702 is undiluted by the RTTuncertainty, and the server's 106 representation of the deterministicprefetch 712 portion of the position 710 can be represented by theserver 106 as a space curve with a specific view direction vector foreach position on the space curve.

As limits are placed on the current position 702 and the position at thesecond location 706, the predictability of navigation is enhanced. Thearea and/or volume of the corresponding current frustum 704 and thesecond frustum 708 are decreased. The decrease in area and/or volume ofthe current frustum 704 and the second frustum 708 further places limitsonto the predicted position of navigation 710, effectively decreasingthe bandwidth required for visibility event packet streaming.

Additionally, the predicted position 710 can be utilized to defer thetransmission of significant portions of the visibility event packets.The visibility event protocol defined by a navigation driven predictiveprefetch of precomputed visibility event packets is an incremental andprogressive method of streaming navigational data. In some aspects ofthe present disclosure, a series of partial and/or deferred visibilityevent packets that reflect predicted sequences of viewcell-to-viewcellboundaries are processed via the circuitry of the server 106. As such,runtime conservative view frustum culling methods are employed in whichsome parts of a particular visibility event packet, corresponding to afirst viewcell boundary, may go untransmitted, even as the positionpenetrates later transited viewcell boundaries.

By having the navigating client device 108 follow the future position ofthe virtual navigational prefetch agent of the server 106 by 2000 ms,the prefetch of visibility event packets essentially becomesdeterministic. This reduces the bandwidth required to stream thevisibility event packets, while enabling the virtual navigationalprefetch agent of the server 106 to respond to changes occurring alongthe intended navigational route by transmitting a new stream ofvisibility event packets corresponding to a diverting route in a timeperiod between 166 ms and 2000 ms, as shown in FIG. 7.

FIG. 8 describes a high-level architectural block diagram of thevisibility event navigation system 800, according to certain aspects.The visibility event navigation system 100 can be attributed to anavigation system on-board a human-operated, autonomous, orsemi-autonomous aircraft or ground vehicle. FIG. 8 illustrates anexemplary navigation system including a client device 108 such as anon-board an autonomous quadrotor aircraft in various stages of flightcorresponding to locations 820, 822, 824 and 826.

The visibility event navigation system 100 can include a network 102including a plurality of network nodes 813, 815 and 839. The network 102including network nodes 813, 815 and 839 can include wireless networkssuch as Wi-Fi, BLUETOOTH, cellular networks including EDGE, 3G and 4Gwireless cellular systems, IMS, or any other wireless form ofcommunication that is known, and may also encompass wired networks, suchas Ethernet.

At the position 820, the client device 108 transmits a request for anavigational route to an intended destination which may be an individualhome, a business building, a porch, a rooftop, an inside of a structure,and the like. Transmission of the request 805 to the server 106 employsa network node 813. The server 106 authenticates the identity of theclient device 108 and transmits 807 initial visibility event navigationdata corresponding to the visibility event packets for viewcells on thenavigational route to the intended destination. The navigational routecan include a portion of the environment represented by a set ofconnected viewcells within the navigable 3D space of the environment anda corresponding model of the environment. Once the initial visibilityevent navigation packets are received at the client device 108, theclient device 108 proceeds from the initial location 820 to location 822along the navigational route to the intended destination. At location822, the client device 108 can be configured to determine locationinformation using data detected at a sensor 104 in communication withthe client device 108. The sensor 104 can be utilized by the clientdevice 108 to determine, directly or indirectly, surface informationincluding the location of the visible 3D surfaces surrounding the clientdevice 108 and compare this surface information to the relevant 3Denvironmental model data delivered by the visibility event navigationalstream. This location information is transmitted to the client device108 through the communication network node 815 in transmission 814.

After receiving the transmission 814 from the client device 108 atposition 822 via the network node 814, the server 106 determines whetherthe client device 108 at location 822 is located along the navigationalroute to the intended destination. If the client device 108 isdetermined to be located along the navigational route to the intendeddestination, then the server transmits 817 additional visibility eventnavigational packets for viewcells further along the navigational routeto the intended destination. In some aspects of the present disclosure,the transmission 817 from the server 106 includes visibility eventnavigation data corresponding to the visibility event packets forviewcells on one or more alternate routes to save landing zones.

FIG. 8 further shows the client device 108 at position 824. Position 824depicts a location that is not on the navigational route to the intendeddestination, but on a route 840 that substantially deviates from thenavigational route. After receiving the transmission 835 from the clientdevice 108 at position 824 via the network node 839, the server 106determines that client device 108 is located on the navigational routeto the intended destination. If the client device 108 is determined tonot be on the navigational route, then the server may transmit a signal837 from the network node 839 to the client device 108 at position 824including a safe landing command. In some aspects, the safe landingcommand causes the client device 108 to utilize the visibility eventpackets for the alternate route to a safe landing zone to navigate alongthe route to the safe landing zone 840 and land at position 826. Incertain aspects, this safe landing command may be implemented as adefault behavior of the visibility even navigation system in instanceswhere communication between the client device 108 and the server 106 isinterrupted for a predetermined period of time.

FIG. 9 is an algorithmic flowchart of a visibility event navigationprocess 900, according to certain exemplary aspects. The visibilityevent navigation process 900 describes a client device 108 navigatingalong a navigational route and the transmission of an alternate routeand safe landing command if the client device 108 strays from thenavigational route. At step 902, the circuitry of the client device 108transmits a request to the server 106 for visibility event packets. Thevisibility event packets can be navigational packets that correspond tothe navigational route that the client device 108 intends to traverse.

At step 904, the server 106 in communication with the client device 108authenticates the request of the client device 108. In certain aspects,the server 106 is configured to deliver visibility event packets tocertified client devices 108 which present authenticated credentialsusing a secure process of authentication, such as a cryptographicticket. The transmission of visibility event packets to certified clientdevices 108 ensures that the client device 108 has met certainrequirements for navigating along the intended route.

At step 906, the circuitry of the server 106 transmits visibility eventnavigational packets to the client device 108. The initial navigationalpackets may include a subset of the entire set of visibility eventpackets corresponding to the contiguous viewcells of the navigable spacerepresenting the specified navigational route to the intendeddestination. In some aspects, the circuitry of the server 106 transmitsvisibility event data representing the set of visibility event packetscorresponding to the contiguous viewcells of the navigable spacerepresenting the route to a safe landing zone.

At step 908, the circuitry of the client device 108 employs the relevantvisibility event packet navigational data and 3D sensor data todetermine location of the client device 108 within the real environmentusing 3D map-matching. In some aspects, the matching includes othermatching methods, computer vision methods, and the like. The matchingmay result in a position relative to the relevant obstacle or targetsurfaces in the real environment. The determination can also result inlatitude, longitude, and/or altitude calculations if the environmentalmodel data of the visibility event packets is geo-registered.

At step 910, the circuitry of the client device 108 transmitsinformation corresponding to the location of the client device 108 inthe real environment to the server 106. In certain aspects, the locationinformation can includes latitude, longitude, and/or altitudecalculations if the environmental model data of the visibility eventpackets is geo-registered.

At step 912, a determination is made of whether the client device 108 islocated along the specific navigational route. In some aspects, theserver 106 determines if the transmitted location of the client device108 is on the navigational route. In certain aspects, the determinationcan employ multiple parameters to determine if the location of thenavigating client device 108 substantially deviates from thenavigational route. For example, the parameters can include the averagedistance from the navigational route over a predetermined period of timein which the desired route is defined as a navigational route bounded byan allowed region of navigational uncertainty reflecting a known ordesired precision in navigation. If the client device is determined tobe located along the navigational route, resulting in a “yes” at step912, the visibility event navigation process 900 proceeds to step 914.Otherwise, if the client device is determined to not be located alongthe navigational route, resulting in a “no” at step 912, the visibilityevent navigation process 900 proceeds to step 918.

At step 914, the circuitry of the server 106 transmits additionalvisibility event navigation packets corresponding to viewcells that arefarther along the specified navigational route to the client device 108.

At step 916, the client device 108 receives the instructions transmittedfrom the server 106 and continues to navigate along the specifiednavigational route. In some aspects, the visibility event navigationprocess 900 ends after completing step 916. In other aspects, thevisibility event navigation process 900 proceeds to step 910.

At step 918, the circuitry of the server 106 transmits a command to theclient device 108 in which the client device 108 is provided withinstructions to navigate on an alternate route towards a safe landingzone and land at the safe landing zone.

At step 920, the client device 108 navigates along the alternate routeand lands at the safe landing zone. In certain aspects, the visibilityevent navigational client executes an internal instruction to follow analternate route to a safe landing zone and land if communication withthe visibility event navigational server is interrupted. In someaspects, the visibility event navigation process 900 ends upon thecompletion of step 920. In other aspects, the visibility eventnavigation process 900 proceeds to step 910.

FIG. 10 is an algorithmic flowchart of a visibility event navigationdifference determination process 1000, according to certain exemplaryaspects. The visibility event navigation difference determinationprocess 1000 describes a client device 108 navigating along anavigational route and the determination of a deviation of the clientdevice 108 from the navigational route. At step 1002, the circuitry ofthe client device 108 transmits a request to the server 106 forvisibility event packets. The visibility event packets can benavigational packets that correspond to the navigational route that theclient device 108 intends to traverse.

At step 1004, the server 106 is in communication with the client device108 and authenticates the request of the client device 108. In certainaspects, the server 106 is configured to deliver visibility eventpackets to certified client devices 108 which present authenticatedcredentials using a secure process of authentication, such as acryptographic ticket. The transmission of visibility event packets tocertified client devices 108 ensures that the client device 108 has metcertain requirements for navigating along the intended route.

At step 1006, the circuitry of the server 106 transmits visibility eventnavigational packets to the client device 108. The initial navigationalpackets may include a subset of the entire set of visibility eventpackets corresponding to the contiguous viewcells of the navigable spacerepresenting the specified navigational route to the intendeddestination.

At step 1008, the circuitry of the client device 108 determines adifference between the environmental model data delivered by thevisibility event packets and the corresponding 3D ground-truth, asdetermined using the 3D sensor data of a sensor 104 that is incommunication with the client device 108. In certain aspects, thedifference determination occurs during the 3D map-matching in which theenvironmental model data supplied by the visibility even packets and the3D data acquired by the sensor 104 determine the location of the clientdevice 108 within the real environment. The difference determination ofstep 1008 may employ one or more statistical metrics of ‘fit’ asdetermined by an iterative closest points algorithm or other matchingalgorithm including variations of the iterative closest points algorithmin which real-time acquired sensor point cloud data is matched to modelpolygons or other surface representations used in the correspondingenvironmental model.

At step 1010, a caution state in navigation is initiated at the clientdevice 108 via the circuitry of the client device 108. In certainaspects, the difference between the representation of the environmentand the ground truth scan from the sensor 104, results in the circuitryutilizing the visibility event packet data with caution in the immediatearea the client device 108 is located at. In this state of cautiousnavigation, the visibility event navigation can employ less precisenavigational methods such as SLAM, GPS, and the like. As such, the moreprecise 3D map-matching or other computer vision based navigation canresume when there is sufficient match between the delivered model dataand the real-time sensor data that does not surpass a predetermineddifference threshold.

At step 1012, the difference information obtained by the circuitry ofthe client device 108 is transmitted from the client device 108 to theserver 106. In certain aspects, the visibility event navigation system100 exploits the fact that the amount of information generally requiredto represent a difference over any modest period, for example in anurban environment, is very small compared to the amount of informationrequired to represent the urban environment. As such, this small amountof difference information can be transmitted as raw point cloud data orcould be first semi-processed, for example into a voxel representation,or into a procedural parametric representation. In some aspects, thedifference information is processed into polygon or procedural surfacesthat can be encoded as visibility event packets for transmission to theserver 106.

At step 1014, a determination is made of whether the differenceinformation is significant and verified. In some aspects, verificationof the difference information can include the server 106 receivingsubstantially the same or similar difference information, for the samecorresponding region of the navigated environment, from a plurality ofclient devices 108 over a predetermined period of time. In certainaspects, the significance of the difference can include spatial metrics,or other metrics that weigh the importance of the change to navigation.For example, the change information can indicate a new structure thatimpedes navigation along one or more navigation routes. In otheraspects, the significance of the change information can include metricsof saliency to the client device 108 including 3D map-matching or othercomputer vision algorithms that use the real surfaces and thecorresponding environmental 3D model data, delivered as visibility eventpackets. If the delivered environmental change information satisfies apredetermined threshold for significance, resulting in a “yes” at step1014, the visibility event navigation difference determination process1000 proceeds to step 1016. Otherwise, if the delivered environmentalchange information does not satisfy a predetermined threshold forsignificance, resulting in a “no” at step 1014, the visibility eventnavigation difference determination process 1000 ends.

At step 1016, the changes to the environmental model are incorporatedand encoded as visibility event packets by the circuitry of the server106.

At step 1018, the visibility event packets are transmitted on anon-demand basis to any subsequent client devices 108 in which thechanged surfaces are unoccluded from the viewcells defining the currentnavigational route. In certain aspects, the transmission employs thesame navigation-based predictive prefetch used to transmit the unchangedpackets. In some aspects, the updated visibility even navigation data isreceived by a second client device. For example, the second clientdevice can include an aircraft or ground vehicle that has entered, orplans to enter regions of the real environment in which the changesdetected by the client device 108 have now been encoded as visibilityevent packets.

FIG. 11 is an algorithmic flowchart of a visibility event navigationincident determination process 1100, according to certain exemplaryaspects. The visibility event navigation incident determination process1100 describes a client device 108 that navigates to a determinedincident location or a location along an ingress/egress route to/fromthe incident location. At step 1102, the circuitry of the client device108 transmits a request to the server 106 for visibility event packets.The visibility event packets can be navigational packets that correspondto the navigational route that the client device 108 intends totraverse.

At step 1104, the server 106 is in communication with the client device108 and authenticates the request of the client device 108. In certainaspects, the server 106 is configured to deliver visibility eventpackets to certified client devices 108 which present authenticatedcredentials using a secure process of authentication, such as acryptographic ticket. The transmission of visibility event packets tocertified client devices 108 ensures that the client device 108 has metcertain requirements for navigating along the intended route.

At step 1106, the circuitry of the server 106 transmits visibility eventnavigational packets to the client device 108. The initial navigationalpackets may include a subset of the entire set of visibility eventpackets corresponding to the contiguous viewcells of the navigable spacerepresenting the specified navigational route to the intendeddestination.

At step 1108, the client device 108 navigates along the specifiednavigational route.

At step 1110, a determination is made of whether an incident hasoccurred. In certain aspects, the circuitry of the server 106 determinesif there has been an incident in the real environment. In some aspects,the incident may be an event that threatens public safety or security.If an incident is determined to be present, resulting in a “yes” at step1110, the visibility event navigation incident determination process1100 proceeds to step 1112. Otherwise, if an incident is not determinedto be present, resulting in a “no” at step 1110, the visibility eventnavigation incident determination process 1100 proceeds to step 1116.

At step 1112, visibility event packets corresponding to a route to theincident location are transmitted from server 106 to the client device108 via the circuitry of the server 106. In certain aspects, thetransmitted visibility event packets can correspond to one or moreingress or one or more egress routes that are to or from the incidentlocation.

At step 1114, the circuitry of the server 106 transmits a command to theclient device 108, which instructs the client device 108 to navigate tothe incident location or to a location along the ingress/egress routesthat are to/from the incident. In certain aspects, the diversion of theclient device 108 to an egress route enables surveillance for incidentperpetrators attempting to escape from the incident location. Asdescribed in copending parent application Ser. No. 13/807,824, themethod of precomputing visibility event packets using conservativelinearized umbral event surfaces can incorporate identifying thenavigable space of the environment and possible routes within themodeled space, including ingress and egress routes. In exemplaryaspects, visibility event packets delivered in step 460 can includethose corresponding to positions from which egress routes and ingressroutes are unoccluded and which provide good vantage points for mobilesurveillance.

At step 1116, the circuitry of the server 106 transmits more visibilityevent navigational packets for the specified route to the client device108. In some aspects, the visibility event navigation incidentdetermination process 1100 proceeds to step 1108 upon the completion ofstep 1116. In other aspects, the visibility event navigation incidentdetermination process 1100 ends upon the completion of step 1116.

FIG. 12 is an algorithmic flowchart of a visibility event navigationunauthorized object determination process 1200, according to certainexemplary aspects. The visibility event navigation unauthorized objectdetermination process 1200 describes a process of detecting andresponding to unauthorized objects in a real environment. At step 1202,the circuitry of the client device 108 transmits a request to the server106 for visibility event packets. The visibility event packets can benavigational packets that correspond to the navigational route that theclient device 108 intends to traverse.

At step 1204, the server 106 is in communication with the client device108 and authenticates the request of the client device 108. In certainaspects, the server 106 is configured to deliver visibility eventpackets to certified client devices 108 which present authenticatedcredentials using a secure process of authentication, such as acryptographic ticket. The transmission of visibility event packets tocertified client devices 108 ensures that the client device 108 has metcertain requirements for navigating along the intended route.

At step 1206, the circuitry of the server 106 transmits visibility eventnavigational packets to the client device 108. The initial navigationalpackets may include a subset of the entire set of visibility eventpackets corresponding to the contiguous viewcells of the navigable spacerepresenting the specified navigational route to the intendeddestination. The circuitry of the server 106 also transmits informationrepresenting the location of authorized client devices 108 operating inthe vicinity of the specified navigational route. In certain aspects,the vicinity includes an area within a predetermined sensor range of thespecified navigational route, wherein the sensor range includes therange of a sensor 104 that is in communication with the client device108.

At step 1208, the circuitry of the client device 108 detectsunauthorized moving objects in the real environment. In certain aspects,the real environment includes airspace. The method of detection caninclude one or more methods of detecting moving objects via on-boardLiDAR, SONAR, Radar, optical sensors, or any other detection means thatare known. In some aspects, utilizing the information describing knownlocations of authorized client devices in the vicinity can improve thedetection of unauthorized moving objects. The client device 108 can alsouse the sensor 104 such an on-board sensor to detect, track, and reportthe location of other authorized client devices 108, or authorizedclient devices 108 that have lost communication with the server 106.

At step 1210, the circuitry of the client device 108 transmitsinformation describing the location of unauthorized moving objects tothe server 106.

At step 1212, a determination is made of whether the detection of amoving object is significant and/or verified. In certain aspects, thedetermination of significance may include size, speed and otherparameters of the moving object. In some aspects, the verification mayinclude repeated, correlated observations from multiple client devices108 or other authenticated sources. If the moving object is determinedto be significant and/or verified, resulting in a “yes” at step 1212,the visibility event navigation unauthorized object determinationprocess 1200 proceeds to step 1214. Otherwise, if the moving object isdetermined to not be significant or verified, resulting in a “no” atstep 1212, the visibility event navigation unauthorized objectdetermination process 1200 ends.

At step 1214, the circuitry of the server 106 transmits the location ofthe unauthorized object such as an aircraft or ground vehicle to aclient device. The circuitry of the server 106 can also be configured totransmit the visibility event packets corresponding to an intercept orpursuit route to the unauthorized object. In some aspects, the server106 can also transmit a command to the client device 108 which instructsthe client device 108 to pursue, track, or otherwise engage the detectedunauthorized object.

At step 1216, the client device 108 initiates pursuit, tracking, orengagement of the unauthorized object by following a pursuit route orintercept route specified by the visibility event packets.

FIG. 13 is a block diagram of a visibility event navigation systemworkflow 1300, according to certain exemplary aspects. The visibilityevent navigation system workflow 1300 describes a high-levelarchitectural block diagram of the visibility event navigation system100. The visibility event navigation system 100 can be used forstreaming visibility event data and other data to navigating clientdevices 108 such as vehicles, aircraft, and the like. In some aspects,the client devices can be autonomous, semi-autonomous, on-board pilotoperated, on-board driver operated, remotely operated, and the like. Theserver 1310 of the visibility event navigation system 100 can includecircuitry that is configured to process and transmit urban and/ornatural environmental model surfaces, content, and the like of a 3Denvironmental model database 1302 to a client device 108 via a network102.

The circuitry of the server 106 can be configured to deliver theenvironmental model surfaces and the content of the 3D environmentalmodel database 1302 via a visibility event data stream employingvisibility event packets 1306 that are encoded using first-ordervisibility propagation via a visibility event packet encoder 1304. Incertain aspects, the visibility event navigation packets 1306 areprecomputed from the 3D environmental model 1302 at an earlier time andstored for later use. The circuitry of the server 1310 can further beconfigured to run a visibility event server software 1308 in which thevisibility event packets 1306 are processed in a visibility eventnavigation server 1310. The processed visibility event packets 1306 ofthe server 1310 can be transmitted to a client device 108 in whichvisibility event client software 1312 employs the processed visibilityevent packets 1306 in a 3D map-matching or computer vision basednavigation system 1314. In certain aspects, the visibility event serversoftware 1308 employs navigation-driven predictive prefetch to transmitthe precomputed visibility event packets 1306. In certain aspects,specifying a defined navigational route and streaming the visibilityevent packets 1306 to client visibility event navigation software 1312increases the predictability of navigation-driven prefetch and reducesthe bandwidth required to stream the packets.

FIG. 14 is a hardware block diagram of a client device, according tocertain exemplary aspects. In FIG. 14, the client device 108 includes aCPU 1400, which performs the processes described above/below. Theprocess data and instructions may be stored in memory 1402. Theseprocesses and instructions may also be stored on a storage medium disk1404 such as a hard drive (HDD) or portable storage medium or may bestored remotely. Further, the claimed advancements are not limited bythe form of the computer-readable media on which the instructions of theinventive process are stored. For example, the instructions may bestored on CDs, DVDs, in FLASH memory, RAM, ROM, PROM, EPROM, EEPROM,hard disk, or any other information processing device with which theclient device 108 communicates, such as a server or computer.

Further, the claimed advancements may be provided as a utilityapplication, background daemon, or component of an operating system, orcombination thereof, executing in conjunction with CPU 1400 and anoperating system such as MICROSOFT WINDOWS, UNIX, SOLARIS, LINUX, APPLEMAC-OS, and other systems known to those skilled in the art.

The hardware elements in order to achieve the server 106 may be realizedby various circuitry elements, known to those skilled in the art. Forexample, CPU 1400 may be a XENON or CORE processor from Intel of Americaor an OPTERON processor from AMD of America, or may be other processortypes that would be recognized by one of ordinary skill in the art.Alternatively, the CPU 1400 may be implemented on an FPGA, ASIC, PLD orusing discrete logic circuits, as one of ordinary skill in the art wouldrecognize. Further, CPU 1400 may be implemented as multiple processorscooperatively working in parallel to perform the instructions of theinventive processes described above.

The client device 108 in FIG. 14 also includes a network controller1406, such as an Intel Ethernet PRO network interface card from IntelCorporation of America, for interfacing with a network 102. As can beappreciated, the network 102 can be a public network, such as theInternet, or a private network such as an LAN or WAN network, or anycombination thereof and can also include PSTN or ISDN sub-networks. Thenetwork 102 can also be wired, such as an Ethernet network, or can bewireless such as a cellular network including EDGE, 3G and 4G wirelesscellular systems. The wireless network can also be Wi-Fi, BLUETOOTH, orany other wireless form of communication that is known.

The client device 108 further includes a display controller 1408, suchas a NVIDIA GeForce GTX or Quadro graphics adaptor from NVIDIACorporation of America for interfacing with display 1410, such as aHewlett Packard HPL2445w LCD monitor. A general purpose I/O interface1412 interfaces with a touch screen panel 1416 on or separate fromdisplay 1410. General purpose I/O interface also connects to a varietyof peripherals 1418 including printers and scanners, such as anOfficeJet or DeskJet from Hewlett Packard.

A sound controller 1420 is also provided in the client device 108, suchas Sound Blaster X-Fi Titanium from Creative, to interface withspeakers/microphone 1422 thereby providing sounds and/or music.

The general purpose storage controller 1424 connects the storage mediumdisk 1404 with communication bus 1426, which may be an ISA, EISA, VESA,PCI, or similar, for interconnecting all of the components of the clientdevice 108. A description of the general features and functionality ofthe display 1410, display controller 1408, storage controller 1424,network controller 1406, sound controller 1420, and general purpose I/Ointerface 1412 is omitted herein for brevity as these features areknown.

The exemplary circuit elements described in the context of the presentdisclosure may be replaced with other elements and structureddifferently than the examples provided herein. Moreover, circuitryconfigured to perform features described herein may be implemented inmultiple circuit units (e.g., chips), or the features may be combined incircuitry on a single chipset, as shown in FIG. 15.

FIG. 15 is a hardware block diagram of a data processing system 1500,according to certain exemplary aspects. FIG. 15 shows a schematicdiagram of a data processing system 1500, for performing visibilityevent navigation. The data processing system 1500 is an example of acomputer in which code or instructions implementing the processes of theillustrative aspects may be located.

In FIG. 15, data processing system 1500 employs a hub architectureincluding a north bridge and memory controller hub (NB/MCH) 1525 and asouth bridge and input/output (I/O) controller hub (SB/ICH) 1520. Thecentral processing unit (CPU) 1530 is connected to NB/MCH 1525. TheNB/MCH 1525 also connects to the memory 1545 via a memory bus, andconnects to the graphics processor 1550 via an accelerated graphics port(AGP). The NB/MCH 1525 also connects to the SB/ICH 1520 via an internalbus (e.g., a unified media interface or a direct media interface). TheCPU Processing unit 1530 may contain one or more processors and even maybe implemented using one or more heterogeneous processor systems.

FIG. 16 is a hardware block diagram of a CPU, according to certainexemplary aspects. FIG. 16 shows one implementation of CPU 1530. In oneimplementation, the instruction register 1638 retrieves instructionsfrom the fast memory 1640. At least part of these instructions arefetched from the instruction register 1638 by the control logic 1636 andinterpreted according to the instruction set architecture of the CPU1530. Part of the instructions can also be directed to the register1632. In one implementation the instructions are decoded according to ahardwired method, and in another implementation the instructions aredecoded according a microprogram that translates instructions into setsof CPU configuration signals that are applied sequentially over multipleclock pulses. After fetching and decoding the instructions, theinstructions are executed using the arithmetic logic unit (ALU) 1634that loads values from the register 1632 and performs logical andmathematical operations on the loaded values according to theinstructions. The results from these operations can be feedback into theregister and/or stored in the fast memory 1640.

According to certain implementations, the instruction set architectureof the CPU 1530 can use a reduced instruction set architecture, acomplex instruction set architecture, a vector processor architecture, avery large instruction word architecture. Furthermore, the CPU 1530 canbe based on the Von Neuman model or the Harvard model. The CPU 1530 canbe a digital signal processor, an FPGA, an ASIC, a PLA, a PLD, or aCPLD. Further, the CPU 1530 can be an x86 processor by Intel or by AMD;an ARM processor, a Power architecture processor by, e.g., IBM; a SPARCarchitecture processor by Sun Microsystems or by Oracle; or other knownCPU architecture.

Referring again to FIG. 15, the data processing system 1500 can includethat the SB/ICH 1520 is coupled through a system bus to an I/O Bus, aread only memory (ROM) 1556, universal serial bus (USB) port 1564, aflash binary input/output system (BIOS) 1568, and a graphics controller1558. PCI/PCIe devices can also be coupled to SB/ICH 1520 through a PCIbus 1562.

The PCI devices may include, for example, Ethernet adapters, add-incards, and PC cards for notebook computers. The Hard disk drive 1560 andCD-ROM 1566 can use, for example, an integrated drive electronics (IDE)or serial advanced technology attachment (SATA) interface. In oneimplementation the I/O bus can include a super I/O (SIO) device.

Further, the hard disk drive (HDD) 1560 and optical drive 1566 can alsobe coupled to the SB/ICH 1520 through a system bus. In oneimplementation, a parallel port 1578 and a serial port 1576 can beconnected to the system bust through the I/O bus. Other peripherals anddevices that can be connected to the SB/ICH 1520 using a mass storagecontroller such as SATA or PATA, an Ethernet port, an ISA bus, a LPCbridge, SMBus, a DMA controller, and an Audio Codec.

The functions and features described herein may also be executed byvarious distributed components of a system. For example, one or moreprocessors may execute these system functions, wherein the processorsare distributed across multiple components communicating in a network.The distributed components may include one or more client and servermachines, which may share processing, in addition to various humaninterface and communication devices (e.g., display monitors, smartphones, tablets, personal digital assistants (PDAs)). The network may bea private network, such as a LAN or WAN, or may be a public network,such as the Internet. Input to the system may be received via directuser input and received remotely either in real-time or as a batchprocess.

The above-described hardware description is a non-limiting example ofcorresponding structure for performing the functionality describedherein.

A number of implementations have been described. Nevertheless, it willbe understood that various modifications may be made without departingfrom the spirit and scope of this disclosure. For example, preferableresults may be achieved if the steps of the disclosed techniques wereperformed in a different sequence, if components in the disclosedsystems were combined in a different manner, or if the components werereplaced or supplemented by other components. The functions, processesand algorithms described herein may be performed in hardware or softwareexecuted by hardware, including computer processors and/or programmablecircuits executing program code and/or computer instructions to executethe functions, processes and algorithms described herein. Additionally,an implementation may be performed on modules or hardware not identicalto those described. Accordingly, other implementations are within thescope that may be claimed.

The above disclosure also encompasses the aspects listed below.

(1) A method of visibility event navigation, including one or morevisibility event packets located at a server, the one or more visibilityevent packets including visibility event packet information representing3D surface elements of a geospatial model that are occluded from a firstviewcell and not occluded from a second viewcell, the first and secondviewcells representing spatial regions of a navigational route within areal environment modeled by the geospatial model, including: receiving,via processing circuitry of a client device, at least one visibilityevent packet of the one or more visibility event packets from theserver; detecting, via the circuitry, surface information representingone or more visible surfaces of the real environment at a sensor incommunication with the client device; calculating, via the circuitry, atleast one position of the client device in the real environment bymatching the surface information to the visibility event packetinformation corresponding to a first visibility event packet of the oneor more visibility event packets; transmitting, via the circuitry, theat least one position from the client device to the server; andreceiving, via the circuitry, at least one second visibility eventpacket of the one or more visibility event packets when the at least oneposition is within the navigational route at the client device from theserver.

(2) The method of (1), further including: receiving, via the circuitry,at least one alternate visibility event packet of the one or morevisibility event packets at the client device from the server, whereinthe at least one alternate visibility event packet representing 3Dsurface elements of the geospatial model that are occluded from a thirdviewcell but not occluded from a fourth viewcell, the third and fourthviewcells representing spatial regions of a an alternate navigationalroute within a real environment modeled by the geospatial model, thealternate navigational route leading to a safe landing zone.

(3) The method of either (1) or (2), further including: receiving, viathe circuitry, a command for navigation to occur along the alternatenavigational route at the client device from the server, when the atleast one position is not within the navigational route.

(4) The method of any one of (1) to (3), further including: receiving,via the circuitry, a command for navigation to occur along the alternatenavigational route and the safe landing zone when the client fails toreceive a signal from the server during a predetermined period of time.

(5) The method of any one of (1) to (4), wherein the client deviceincludes a navigation system for at least one of an autonomous aircraft,a semiautonomous aircraft and a piloted aircraft.

(6) The method of any one of (1) to (5), wherein the safe landing zoneincludes a specific viewcell as a zone reachable by the autonomousaircraft in case of an engine failure while the autonomous aircraft isin the specific viewcell.

(7) The method of any one of (1) to (6), wherein the safe landing zoneincludes a specific viewcell as a zone reachable by the piloted aircraftin case of an engine failure while the piloted aircraft is in thespecific viewcell.

(8) The method of any one of (1) to (7), further including: detecting,via the circuitry, at least one moving object using the surfaceinformation and the visibility event packet information; andtransmitting, via the circuitry, information representing the locationof the at least one moving object from the client device to the server.

(9) The method of any one of (1) to (8), wherein the real environment isan urban environment, and the geospatial model is a model of the urbanenvironment.

(10) The method of any one of (1) to (9), wherein the real environmentis and indoor environment, and the geospatial model is a model of theindoor environment.

(11) The method of any one of (1) to (10), wherein the client deviceincludes a navigation system for at least one of an autonomous groundvehicle and a semiautonomous ground vehicle.

(12) The method of any one of (1) to (11), wherein the client device isa navigational system for a piloted aircraft and the visibility eventpacket include representations of obstacles hazardous to aviation.

(13) The method of any one of (1) to (12), further including: receivingone or more visibility event packets when a request for the one or morevisibility event packets is authenticated by the server.

(14) A method of visibility event navigation, including one or morevisibility event packets located at a server, including informationrepresenting 3D surface elements of a geospatial model that are occludedfrom a first viewcell and not occluded from a second viewcell, the firstand second viewcells representing spatial regions of a navigationalroute within a real environment modeled by the geospatial model,including: prefetching, via processing circuitry of the server, a firstvisibility event packet of the one or more visibility event packets to aclient device; receiving, via the circuitry, at least one position ofthe client device in the real environment at the server; andtransmitting, via the circuitry, a second visibility event packet of theone or more visibility event packets to the client device when the atleast one position is within the navigational route.

(15) The method of (14), further including: transmitting, via thecircuitry, at least one alternate visibility event packet of the one ormore visibility event packets from the server to the client, the atleast one alternate visibility event packet representing 3D surfaceelements of the geospatial model that are occluded from a third viewcellbut not occluded from a fourth viewcell, the third and fourth viewcellsrepresenting spatial regions of an alternate navigational route within areal environment modeled by the geospatial model, the alternatenavigational route leading to a safe landing zone.

(16) The method of either (14) or (15), wherein the client device is anavigational system includes at least one of an autonomous aircraft anda semiautonomous aircraft.

(17) The method of any one of (14) to (16), further including:receiving, via the circuitry, at least one object position in the realenvironment of a moving object at the server from the client device.

(18) The method of any one of (14) to (17), further including:transmitting, via the circuitry, a command for navigation to occur alongthe alternate navigational route from the server to the client device,when the at least one position is not within the specified navigationalroute.

(19) The method of any one of (14) to (18), wherein the real environmentis an urban environment, and the geospatial model is a model of theurban environment.

(20) The method of any one of (14) to (19), wherein the real environmentis an indoor environment, and the geospatial model is a model of theindoor environment.

(21) A method of visibility event navigation prefetch, including atleast one partial visibility event packet including a subset of acomplete visibility event packet, the complete visibility event packetincluding information representing 3D surface elements of a geospatialmodel occluded from a first viewcell and not occluded from a secondviewcell, the first and second viewcells representing spatial regions ofa navigational route within a real environment modeled by the geospatialmodel, including: receiving, via processing circuitry of a server,information at the server from a client device representing anorientation of a sensor located at the client device, the sensoracquiring information representing the visible surfaces of the realenvironment; and transmitting, via the circuitry, the at least onepartial visibility event packet from the server to the client device,wherein the at least one partial visibility event packet intersects amaximal view frustum including a volume of space intersected by the viewfrustum of the sensor during movement of the client device in the secondviewcell.

(22) A method of visibility event navigation, including at least onepartial visibility event packet located at a server, the at least onepartial visibility event packet including a subset of a completevisibility event packet, the complete visibility event packet includingvisibility event packet information representing 3D surface elements ofa geospatial model occluded from a first viewcell and not occluded froma second viewcell, the first and second viewcells representing spatialregions of a navigational route within a real environment modeled by thegeospatial model, including: transmitting, via processing circuitry of aclient device, surface information from the client device to the servercorresponding to the orientation of a sensor located at the clientdevice, the surface information representing visible surfaces of thereal environment; and receiving, via the circuitry, the at least onepartial visibility event packet at the client device from the serverincluding a subset of the visibility event packet information thatintersects a maximal view frustum, wherein the maximal view frustumincludes a volume of space intersected by a view frustum of the sensorduring movement of the client device in the second viewcell.

(23) A method of visibility event navigation, including a firstvisibility event packet of one or more visibility event packets from aserver, the one or more visibility event packets including visibilityevent packet information representing 3D surface elements of ageospatial model that are occluded from a first viewcell and notoccluded from a second viewcell, the first and second viewcellsrepresenting spatial regions of a navigational route within a realenvironment modeled by the geospatial model, including: detecting, viaprocessing circuitry of a first client device of a plurality of clientdevices, surface information representing visible surfaces of the realenvironment at a sensor in communication with the first client device ofthe plurality of client device; calculating, via the circuitry, at leastone position of the first client device of the plurality of clientdevices in the real environment by matching the surface information tothe visibility event packet information; transmitting, via thecircuitry, the at least one position from the first client device of theplurality of client devices to the server; receiving, via the circuitry,at least one second visibility event packet of the one or morevisibility event packets at the first client device of the plurality ofclient devices from the server when the at least one position is withinthe navigational route; detecting, via the circuitry, positioninformation representing the position of at least one second clientdevice of the one or more client devices in the real environment at thesensor; and transmitting, via the circuitry, the position informationfrom the first client device of the plurality of client devices to theserver.

(24) A method of visibility event navigation prefetch, including a firstvisibility event packet of one or more visibility event packets, the oneor more visibility event packets including visibility event packetinformation representing 3D surface elements of a geospatial model thatare occluded from a first viewcell and not occluded from a secondviewcell, the first and second viewcells representing spatial regions ofa navigational route within a real environment modeled by the geospatialmodel, including: receiving, via processing circuitry of a server, atleast one position of a client device in the real environment at theserver from the client device; and transmitting, via the circuitry, asecond visibility event packet of the one or more visibility eventpackets when the at least one position of the client device is withinthe navigational route and a fee has been paid by an operator of theclient device.

(25) A method of visibility event navigation, including one or morevisibility event packets located at a server, the one or more visibilityevent packets including visibility event packet information representing3D surface elements of a geospatial model that are occluded from a firstviewcell and not occluded from a second viewcell, the first and secondviewcells representing spatial regions of a navigational route within areal environment modeled by the geospatial model, including: receiving,via processing circuitry of a client device, at least one visibilityevent packet of the one or more visibility event packets from theserver; detecting, via processing circuitry of the client device,surface information representing one or more visible surfaces of thereal environment at a sensor in communication with the client device;calculating, via the circuitry, at least one position of the clientdevice in the real environment by matching the surface information tothe visibility event packet information corresponding to the firstvisibility event packet of the one or more visibility event packets;transmitting, via the circuitry, the at least one position in the realenvironment from the client device to the server; receiving, via thecircuitry, at least one second visibility event packet from the serverof the one or more visibility event packets at the client device fromthe server; calculating, via the circuitry, at least one deviation ofthe ground-truth 3D structure from the corresponding environment modeledby the geospatial model using the surface information and the visibilityevent packet information; and transmitting, via the circuitry, the atleast one deviation from the client device to the server.

(26) A visibility event navigation system, including: a server; at leastone client device located in a real environment and in communicationwith the server, the at least one client device including processingcircuitry configured to: detect surface information representing one ormore visible surfaces of the real environment at one or more sensors incommunication with the at least one client device, calculate at leastone position of the at least one client device in the real environmentby matching the surface information to visibility event packetinformation including a first visibility event packet of one or morevisibility event packets representing 3D surface elements of ageospatial model that are occluded from a first viewcell and notoccluded from a second viewcell, the first and second viewcellsrepresenting spatial regions of a navigational route within the realenvironment and modeled by the geospatial model, transmit the at leastone position of the client device to the server, and receive a secondvisibility event packet of the one or more visibility event packets fromthe server when the at least one position is within the navigationalroute.

1. A method of visibility event navigation, including one or morevisibility event packets located at a server, the one or more visibilityevent packets including visibility event packet information representing3D surface elements of a geospatial model that are occluded from a firstviewcell and not occluded from a second viewcell, the first and secondviewcells representing spatial regions of a navigational route within areal environment modeled by the geospatial model, comprising: receiving,via processing circuitry of a client device, at least one visibilityevent packet of the one or more visibility event packets from theserver; detecting, via the circuitry, surface information representingone or more visible surfaces of the real environment at a sensor incommunication with the client device; calculating, via the circuitry, atleast one position of the client device in the real environment bymatching the surface information to the visibility event packetinformation corresponding to a first visibility event packet of the oneor more visibility event packets; transmitting, via the circuitry, theat least one position from the client device to the server; andreceiving, via the circuitry, at least one second visibility eventpacket of the one or more visibility event packets when the at least oneposition is within the navigational route at the client device from theserver.